Uplift capacity of horizontal anchor plate in geocell reinforced sand

The paper investigates the uplift performance of horizontal anchor plate in geocell reinforced sand through a series of model tests. It is noted that the unreinforced anchor plate undergoes a clear failure at a displacement of about 3% of its width, whereas with the provision of geocell and a layer of geotextile right below the geocell mattress significantly increases the uplift capacity by about 4.5 times higher than that of unreinforced sand and could sustain anchor displacement of more than 60%. Results indicates that the geocell mattress by virtue of its rigidity distributes the uplift load in the lateral directions to a larger area, thereby reducing the stress in the overlying soil mass and hence increases the performance of anchor plate system. The provision of the additional geotextile layer right below the geocell mattress is found to be very effective in increasing the stiffness as well as load carrying capacity of anchor plate system. The optimum size (i.e., width and length) of geocell mattress giving adequate load carrying capacity of anchor plate is found to be 5.4 times of anchor width (5.4B). The comparison of model tests results with 3D numerical analysis shows good agreement, indicating that the proposed model is able to capture the uplift load-displacement behaviour of geocell reinforced anchor plate system.     

基于强化学习的HP模型优化方法研究

蛋白质结构预测问题一直是生物信息学中的重要问题。基于疏水极性模型的蛋白质二维结构预测问题是一个典型的NP难问题。目前疏水极性模型优化的方法有贪心算法、粒子群算法、遗传算法、蚁群算法和蒙特卡罗模拟方法等，但这些方法成功收敛的鲁棒性不高，容易陷入局部最优。由此提出一种基于强化学习的HP模型优化方法，利用其连续马尔可夫最优决策与最大化全局累计回报的特点，在全状态空间中，构建基于能量函数的奖赏函数，引入刚性重叠检测规则，充分挖掘生物序列中的全局进化关系，从而进行有效与稳定的预测。以3条经典论文序列和5条Uniref50序列为实验对象，与贪心算法和粒子群算法分别进行了鲁棒性、收敛性与运行时间的比较。贪心算法只能在62.5%的序列上进行收敛，该文方法能在5万次训练后稳定的在所有序列上达到了收敛。与粒子群算法相比，两者都能找到最低能量结构，但该文的运行时间较粒子群算法降低了63.9%。     

基于优势学习的深度Q网络

强化学习问题中，同一状态下不同动作所对应的状态-动作值存在差距过小的现象，Q-Learning算法采用MAX进行动作选择时会出现过估计问题，且结合了Q-Learning的深度Q网络（Deep Q Net）同样存在过估计问题。为了缓解深度Q网络中存在的过估计问题，提出一种基于优势学习的深度Q网络，通过优势学习的方法构造一个更正项，利用目标值网络对更正项进行建模，同时与深度Q网络的评估函数进行求和作为新的评估函数。当选择的动作是最优动作时，更正项为零，不对评估函数的值进行改动，当选择的动作不是最优动作时，更正项的值为负，降低了非最优动作的评估值。和传统的深度Q网络相比，基于优势学习的深度Q网络在Playing Atari 2600的控制问题breakout、seaquest、phoenix、amidar中取得了更高的平均奖赏值，在krull、seaquest中取得了更加稳定的策略。     

Deep reinforcement learning-based cooperative interactions among heterogeneous vehicular networks

Most real-world vehicle nodes can be structured into an interconnected network of vehicles. Through structuring these services and vehicle device interactions into multiple types, such internet of vehicles becomes multidimensional heterogeneous overlay networks. The heterogeneousness of the overlays makes it difficult for the overlay networks to coordinate with each other to improve their performance. Therefore, it poses an interesting but critical challenge to the effective analysis of heterogeneous virtual vehicular networks. A variety of virtual vehicular networks can be easily deployed onto the native network by applying the concept of SDN (Software Defined Networking). These virtual networks reflect their heterogeneousness due to their different performance goals, and they compete for the same physical resources of the underlying network, so that a sub-optimal performance of the virtual networks may be achieved. Therefore, we propose a Deep Reinforcement Learning (DRL) approach to make the virtual networks cooperate with each other through the SDN controller. A cooperative solution based on the asymmetric Nash bargaining is proposed for co-existing virtual networks to improve their performance. Moreover, the Markov Chain model and DRL resolution are introduced to leverage the heterogeneous performance goals of virtual networks. The implementation of the approach is introduced, and simulation results confirm the performance improvement of the latency sensitive, loss-rate sensitive and throughput sensitive heterogeneous vehicular networks using our cooperative solution.     

Stress distribution in thick-walled cylinder due to non-uniform radial pressure on the example of reinforcement corrosion in concrete

This paper presents an analytical solution to the non-uniform pressure on thick-walled cylinder. The formulation is based on the linear elasticity theory (plain strain) and stress function method. As an example, the proposed solution is used to model the stress distribution due to non-uniform steel reinforcement corrosion in concrete. The model is formulated considering different scenarios of corrosion pressure distribution. It is validated against the finite element model for different cases of non-uniform pressure distributions. The results show that the corrosion-induced cracks are likely to start just beyond the anodic zone. This is confirmed by the experimental tests on concrete cylinder exposed to non-uniform accelerated corrosion of steel reinforcement. The model can be effectively used to calculate the distribution of corrosion-induced stresses in concrete.     

Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision

In an environment where robots coexist with humans, mobile robots should be human-aware and comply with humans' behavioural norms so as to not disturb humans' personal space and activities. In this work, we propose an inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision. In this method, the planning process of time-dependent A* is regarded as a Markov decision process and the cost function of the time-dependent A* is learned using the inverse reinforcement learning via capturing humans' demonstration trajectories. With this method, a robot can plan a path that complies with humans' behaviour patterns and the robot's kinematics. When constructing feature vectors of the cost function, considering the local vision characteristics, we propose a visual coverage feature for enabling robots to learn from how humans move in a limited visual field. The effectiveness of the proposed method has been validated by experiments in real-world scenarios: using this approach robots can effectively mimic human motion patterns when avoiding pedestrians; furthermore, in a limited visual field, robots can learn to choose a path that enables them to have the larger visual coverage which shows a better navigation performance.     

Intelligent scheduling of discrete automated production line via deep reinforcement learning

The reinforcement learning (RL) is being used for scheduling to improve the adaptability and flexibility of an automated production line. However, the existing methods only consider processing time certain and known and ignore production line layouts and transfer unit, such as robots. This paper introduces deep RL to schedule an automated production line, avoiding manually extracted features and overcoming the lack of structured data sets. Firstly, we present a state modelling method in discrete automated production lines, which is suitable for linear, parallel and re-entrant production lines of multiple processing units. Secondly, we propose an intelligent scheduling algorithm based on deep RL for scheduling automated production lines. The algorithm establishes a discrete-event simulation environment for deep RL, solving the confliction of advancing transferring time and the most recent event time. Finally, we apply the intelligent scheduling algorithm into scheduling linear, parallel and re-entrant automated production lines. The experiment shows that our scheduling strategy can achieve competitive performance to the heuristic scheduling methods and maintains stable convergence and robustness under processing time randomness.     

回采巷道过断层施工技术

受断层影响,巷道过断层期间掘进速度缓慢,影响采掘接替和矿井生产效率。为了更好地促进矿井生产,以23上04轨顺掘进为研究对象,采用FLAC 3D软件模拟巷道过断层时巷道围岩变形量和塑性区范围,并将围岩划分为正常围岩、过渡带和破碎带3部分。根据模拟结果可知,围岩变形量和塑性区范围都在过渡带开始增大,在破碎带达到最大值。巷道采用后退卧底法过断层,并对断层破碎带及时采用锚网索、注浆、U型钢棚进行支护,通过联合支护相互配合,可以共同提高巷道支护强度,增强巷道顶帮抗变形能力。     

Nonlinear torsional buckling and postbuckling of eccentrically stiffened FGM cylindrical shells in thermal environment

The main aim of this paper is to investigate the nonlinear buckling and post-buckling of functionally graded stiffened thin circular cylindrical shells surrounded by elastic foundations in thermal environments and under torsional load by analytical approach. Shells are reinforced by closely spaced rings and stringers in which material properties of shell and the stiffeners are assumed to be continuously graded in the thickness direction. The elastic medium is assumed as two-parameter elastic foundation model proposed by Pasternak. Based on the classical shell theory with von Karman geometrical nonlinearity and smeared stiffeners technique, the governing equations are derived. Using Galerkin method with three-term solution of deflection, the closed form to find critical torsional load and post-buckling load–deflection curves are obtained. The effects of temperature, stiffener, foundation, material and dimensional parameters are analyzed.